Conventionally, a technique for removing noise components from a digital image on which noise components that are different from signal components are superposed has been studied. The characteristics of noise are diverse depending on their generation factors, and noise removal methods suited to those characteristics have been proposed. For example, when an image input device such as a digital camera, image scanner, or the like is assumed, noise components are roughly categorized into noise which depends on the input device characteristics of a solid-state image sensing element or the like and input conditions such as an image sensing mode, scene, or the like, and has already been superposed on a photoelectrically converted analog original signal, and noise which is superposed via various digital signal processes after the analog signal is converted into a digital signal via an A/D converter.
As an example of the former, impulse noise that generates an isolated value to have no correlation with surrounding image signal values, noise resulting from the dark current of the solid-state image sensing element, and the like are known. As an example of the latter, noise components are amplified simultaneously with signal components when a specific density, color, and the like are emphasized in various correction processes such as gamma correction, gain correction for improving the sensitivity, and the like, thus increasing the noise level. As an example of deterioration due to a digital signal process, since an encoding process using a JPEG algorithm extracts a plurality of blocks from two-dimensional (2D) image information, and executes orthogonal transformation and quantization for respective blocks, a decoded image suffers block distortion that generates steps at the boundaries of blocks.
In addition to various kinds of noise mentioned above, a factor that especially impairs the image quality is noise (to be referred to as “low-frequency noise” hereinafter) which is generated in a low-frequency range and is conspicuously observed in an image sensed by a digital camera or the like. This low-frequency noise often results from the sensitivity of a CCD or CMOS sensor as a solid-state image sensing element. In an image sensing scene such as a dark scene with a low signal level, a shadowy scene, or the like, low-frequency noise is often emphasized due to gain correction that raises signal components irrespective of poor S/N ratio. Furthermore, the element sensitivity of the solid-state image sensing element depends on its chip area. Hence, in a digital camera which has a large number of pixels within a small area, the amount of light per unit pixel consequently decreases, and the sensitivity lowers, thus producing noise. Such low-frequency noise is often visually recognized as pseudo speckle texture across several to ten-odd pixels on a flat portion such as a sheet of blue sky or the like. Some digital cameras often produce false colors.
As a conventionally proposed noise removal method, a method using a median filter, and a method using a low-pass filter (to be abbreviated as LPF hereinafter) that passes only a low-frequency range through it are prevalent.
As the method using a median filter, many methods such as a method which extracts a pixel value which assumes a median (central value) from those of a pixel of interest and its surrounding pixels, and replaces the pixel value of interest by the extracted value (for example, see patent reference 1: Japanese Patent Laid-Open No. 4-235472) and the like have been proposed. If the pixel of interest is impulse noise or random noise, the median filter can replace the value of the pixel of interest, which is an isolated value having low correlation with surrounding pixels, by a median having high correlation, thus removing the isolated value produced in original information.
The method using an LPF calculates the weighted mean using the pixel values of a pixel of interest and its surrounding pixels, and replaces the pixel value of interest by the calculated weighted mean, as shown in, e.g., FIG. 27. This method is mainly effective for the aforementioned block distortion and the like. Since block distortion is noise that generates block-like steps which are different from signal components in a flat region, these steps can become hard to be visually recognized by moderating the slope of the steps.
Note that the aforementioned two methods are locally effective, but blur an edge portion. For this reason, many modifications of these methods have been proposed. For example, a method that makes product sum calculations by selecting only surrounding pixels which have pixel values approximate to that of the pixel of interest upon calculating the weighted mean, so as to prevent any blur due to a filter process (for example, see patent reference 2: Japanese Patent Laid-Open No. 2000-245179) has been proposed.
In addition to the aforementioned method using a median filter and the method using an LPF, many proposals for removing noise and distortion have been made. For example, in order to remove block distortion, a method that exchanges signal values between pixels which are located on the two sides of a block boundary, (for example, see patent reference 3: Japanese Patent Laid-Open No. 8-56357) and a method that adds a predetermined pattern, which is selected from a plurality of patterns using a random number, to pixel signal levels around a block boundary (for example, see patent reference 4: Japanese Patent Laid-Open No. 10-98722) have been disclosed. Also, a method that removes block distortion which is produced upon encoding by adding an error to the level value of a specific pixel of interest having a block boundary as the center (for example, see patent reference 5: Japanese Patent Laid-Open No. 7-75103), and a method that removes noise by detecting maximum and minimum values from neighboring pixels of a pixel of interest and selecting one of the maximum value, minimum value, and value of the pixel of interest in accordance with a determination result indicating if noise is contained as a control signal, so as to remove white and black points that assume isolated values (for example, see patent reference 6: Japanese Patent Laid-Open No. 4-239886) have been disclosed.
In recent years, image processes such as noise removal are normally executed using a host computer such as a personal computer (PC). Recently, however, a recording apparatus (direct recording apparatus) which executes image processes by itself without any host computer, and prints images (for example, see patent reference 7: Japanese Patent No. 3161427) is also available.
However, none of the above prior arts are effective for low-frequency noise such as speckle noise or the like. Especially, the method using a median filter provides only an effect of removing an isolated value having low correlation with surrounding pixel values, and the method using an LPF is effective for only high-frequency noise or random white noise, since it cuts off a high-frequency range. Hence, low-frequency noise cannot be reduced.
The method of patent reference 3 that aims at removal of block distortion can expect an effect of eliminating steps by a method based on random number addition and a method that exchanges pixel values between blocks, as long as a block boundary is known, since the block distortion is high-frequency components produced as rectangular steps. However, low-frequency noise is coupled noise in which pixel values having nearly no change continuously appear over a broad range from several pixels to ten-odd pixels, and the block distortion reduction technique cannot be directly applied. Of course, position information of noise is not known unlike a block boundary upon block encoding.
Since the aforementioned method based on random number addition applies a pixel value which is not present in surrounding pixels, if a random number is added to each of color-separated color components in a color image, a new color which is not present in surrounding pixels is generated, thus causing deterioration of image quality such as false color generation and the like.
Note that a method which reduces electric power of a specific frequency (for example, see patent reference 8: Japanese Patent Laid-Open No. 7-203210) has been disclosed, although its purpose is different from the aforementioned noise removal. In a system which inputs a dot pattern document and executes dithering of a pseudo halftone process upon outputting an image, moiré is generated due to interference between the frequency of the input dot pattern and dithering frequency. In order to prevent generation of such moiré, the invention described in patent reference 8 discloses a moiré removal method that removes the frequency of the input dot pattern in advance. In order to remove the frequency of the dot pattern, it is effective to disturb a predetermined regularity. Hence, in this method, the value of a pixel of interest is exchanged by a pixel value which is located the predetermined number of pixels ahead of the pixel of interest in one dimension. In the above disclosure, the predetermined number of pixels is either fixed or randomly selected.
However, this moiré removal method is not effective for low-frequency noise which is produced broadly in a low-frequency range since it aims at disturbing a specific period corresponding to a peak. Also, exchange of pixel values can preserve the density, but it merely spatially shift pixel phases since the selected pixel value is also changed. A change in selected pixel value corresponds to cyclic filter characteristics, and an impulse response becomes infinite. Even when the predetermined number of pixels as the distance between the pixels to be exchanged is randomly selected, since sampled pixels are exchanged in turn, the moiré period is merely shifted by shifting the phase of the peak of the dot pattern, which is generated at a specific period.
That is, none of the inventions described in the above patent references can effectively reduce noise produced in a low-frequency range.
A method of effectively reducing noise produced in a low-frequency range is disclosed in, e.g., patent reference 9: Japanese Patent Application No. 2002-164625. In this method, a pixel is selected from a window that refers to a pixel of interest and its surrounding pixels of an input image, and a new pixel value is determined based on the selected pixel and the pixel of interest, thus outputting the new pixel value. Hence, this method can effectively reduce noise produced in a low-frequency range.
However, upon practicing the invention of patent reference 9 in the aforementioned direct recording apparatus that records without any host computer, the cost often increases since a larger memory size is required. The invention of patent reference 9 requires a larger memory size as the number of surrounding pixels to be referred to increases, since it refers to a pixel of interest and its surrounding pixel of an input image. Since the direct recording apparatus has a smaller memory size than a host computer, the invention of patent reference 9 occupies a larger memory area when it is practiced. In some cases, an increase in memory size is required.